In order to ensure the security of infrared image in infrared imaging systems and avoid the shortages of traditional image encryption method such as low security and poor real-time performance, a new encryption scheme for infrared image by using Lorenz chaotic system was proposed based on the analysis of bit plane features of infrared image. In this scheme, based on the influence factor statistical characters of bit plane of infrared image, the abscissa, ordinate and bit plane of image's higher four bit planes were scrambled through one operation by using Lorenz chaotic system. This scheme extended image encryption form the pixel level to the bit level. Compared to traditional image encryption method, the scheme is based on the special bit plane statistical character of infrared image, and has fast encryption speed and good performance. It can resist exhaustive attack effectively and has good reliability and scrambling degree. The scheme can be applied to the infrared monitoring system with high security requirements to improve system security and prevent hacker intrusion effectively.